Display control device, display control method, and display control program

ABSTRACT

A display control device including: a display control unit configured to control display, on a display unit, of highlighting corresponding to a state of an image depending on an in-focus state of a subject in the image being within a predetermined in-focus near range.

TECHNICAL FIELD

The present technology relates to a display control device, a displaycontrol method, and a display control program.

BACKGROUND ART

In the manual focusing of a digital camera, a photographer searches fora focus position where a subject becomes sharpest on a monitoring imageby adjusting the focus position of a lens, thereby performing a focusingoperation. An assist function called peaking is developed to improve theaccuracy of this focusing (Patent Literature 1).

In the peaking, a marker serving as an auxiliary signal is displayed ina sharp portion of the monitoring image. A photographer is able toadjust focusing easily by searching for a focus position where thelargest number of markers is displayed on a subject. Compared to thesharpness of the image, the marker has higher visibility than thesharpness of the image, so the peaking function has an effect offacilitating the focusing operation.

In the peaking, it is determined that the portion where the energy of ahigh frequency component exceeds a threshold in the input image is infocus, and a marker is rendered to a pixel determined to be in focus.

CITATION LIST Patent Literature

Patent Literature 1: JP 2010-114556A

DISCLOSURE OF INVENTION Technical Problem

However, the magnitude of the energy of a high frequency component inthe monitoring image is affected not only by the degree of focusing butalso by frequency characteristics of a subject itself. In one example,in a subject having high contrast and texture with edge, the highfrequency component tends to be high. Conversely, in a subject havinglow contrast and smooth texture, the high frequency component tends tobe low. In the former case, displayed peaking markers will be saturatedbefore achieving focusing. In the latter case, there is no markerdisplayed even if focusing is achieved. In either case, the marker isnot displayed properly, so there is a problem that the peaking functionfails to assist the focus operation.

The present technology is made in view of such problems, and is intendedto provide a display control device, display control method, and displaycontrol program, capable of performing optimum peaking regardless ofsubjects, shooting conditions, or the like.

Solution to Problem

To solve the above-described problem, a first technique is a displaycontrol device including: a display control unit configured to controldisplay, on a display unit, of highlighting corresponding to a state ofan image depending on an in-focus state of a subject in the image beingwithin a predetermined in-focus near range.

Further, a second technique is a display control method including:controlling display, on a display unit, of highlighting corresponding toa state of an image depending on an in-focus state of a subject in theimage being within a predetermined in-focus near range.

Moreover, a third technique is a display control program causing acomputer to execute a display control method including controllingdisplay, on a display unit, of highlighting corresponding to a state ofan image depending on an in-focus state of a subject in the image beingwithin a predetermined in-focus near range.

Advantageous Effects of Invention

According to the present technology, it is possible to perform optimumpeaking regardless of subjects, shooting conditions, or the like.Moreover, the effect described above is not necessarily limitative andmay be any effect described in this specification.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating a configuration of an imagecapturing device according to the present technology.

FIG. 2 is a diagram illustrated to describe an overview of peakingprocessing.

FIG. 3 is a flowchart illustrating a procedure of peaking processingaccording to a first embodiment.

FIG. 4 is a diagram illustrated to describe settings of a peakingthreshold.

FIGS. 5A, 5B, and 5C are diagrams illustrated to describe the peakingprocessing according to the first embodiment.

FIG. 6 is a flowchart illustrating a procedure of peaking processingaccording to a second embodiment.

FIGS. 7A, 7B, and 7C are diagrams illustrated to describe the peakingprocessing according to the second embodiment.

FIG. 8 is a diagram illustrating an overview of peaking processingaccording to a third embodiment.

FIG. 9 is a flowchart illustrating a procedure of the peaking processingaccording to the third embodiment.

FIG. 10 is a diagram illustrated to describe the peaking processingaccording to the third embodiment.

FIGS. 11A and 11B are diagrams illustrated to describe factorcharacteristics according to a modified example.

FIG. 12 is a block diagram illustrating a configuration of an imagecapturing device according to the modified example.

MODE(S) FOR CARRYING OUT THE INVENTION

Embodiments of the present technology are described below with referenceto the drawings. Moreover, the description is given in the followingorder.

<1. First Embodiment> [1-1. Configuration of Image Capturing Device][1-2. Peaking Processing] <2. Second Embodiment> [2-1. PeakingProcessing] <3. Third Embodiment> [3-1. Peaking Processing] <4. ModifiedExample> 1. First Embodiment [1-1. Configuration of Image CapturingDevice]

The configuration of an image capturing device 100 equipped withfunctions of a display control device according to a first embodiment isnow described. FIG. 1 is a block diagram illustrating the configurationof the image capturing device 100.

The image capturing device 100 includes a control unit 110, an opticalimage capturing system 120, a lens driving driver 130, an image sensor140, a signal processing large-scale integration (LSI) 150, apre-processor 151, an image signal processor 152, a codec 153, a displaycontrol unit 154, a storage unit 160, a display unit 170, an input unit180, and a camera shake sensor 190.

The control unit 110 includes a central processor (CPU), a random-accessmemory (RAM), a read-only memory (ROM), or the like. The ROM stores aprogram or the like that is read and operated by the CPU. The RAM isused as a work memory of the CPU. The CPU executes various kinds ofprocessing in accordance with a program stored in the ROM and issues acommand to control the entire image capturing device 100.

The optical image capturing system 120 includes a lens 121, a halfmirror 122, and a phase difference sensor 123. The lens 121 is used tocondense light from a subject on the image sensor 140. The half mirror122 is used to guide light incident through the lens 121 to the imagesensor 140 and the phase difference sensor 123. The phase differencesensor 123 is a sensor for auto focus (AF) based on phase differencedetection. In the present technology, the phase difference sensor 123acquires a focus evaluation value and an in-focus state is determinedusing this focus evaluation value. In addition, the optical imagecapturing system 120 is configured to further include, in one example, adriving mechanism, a shutter mechanism, and an iris mechanism, which areused to move the photographic lens 121 to perform focusing and zooming.In the present technology, the focus evaluation value refers to theradius of the magnitude of blur in an image, and the magnitude of theblur is represented in pixel units. The blur refers to a portion orstate that is out of focus and blurred in an image. In addition, themagnitude of blur refers typically to the magnitude of spreading of animage (point spread function: PSF, or point spread characteristics) onan image capturing surface when the lens forms an image of a subject ofa point light source. In addition, the best focus condition refers to acondition that the diameter of the PSF becomes the minimum, and at thistime, the lens forms the sharpest image. In addition, the in-focus staterefers to a state in which the PSF is less than a permissible circle ofconfusion. The permissible circle of confusion circle diameter refers tothe maximum diameter of blur that an end viewer fails to perceive theoccurrence of blur in considering a system from image capturing toviewing.

The lens 121 is a lens for condensing light from a subject on the imagesensor 140. The optical image of a subject, which is obtained throughthe lens 121, is guided toward the image sensor 140 by the half mirror122 and is formed on the image sensor 140.

The lens driving driver 130 is composed of, in one example, amicrocomputer or the like, and controls operations of the drivingmechanism, the shutter mechanism, the iris mechanism, or the like of theoptical image capturing system 120 under the control of the control unit110. This allows exposure time (shutter speed), aperture value(F-number), or the like to be adjusted. In addition to the focusevaluation value from the phase difference sensor 123, the positioninformation of the lens 121 that is obtained from the lens drivingdriver 130 is also possible to be used for determining the in-focusstate.

The image sensor 140 photoelectrically converts incident light from asubject into electric charge and outputs it as an analog image capturingsignal. The analog image capturing signal is output from the imagesensor 140 to the pre-processor 151. An example of the image sensor 140includes charge-coupled device (CCD), complementary-metal-oxidesemiconductor (CMOS), or the like. Moreover, the image sensor 140 may beequipped with the function of the phase difference sensor 123.

The pre-processor 151 performs sample-and-hold processing or the like onthe image capturing signal that is output from the image sensor 140 sothat a satisfactory signal-to-noise (S/N) ratio may be maintained bycorrelated double sampling (CDS) processing. Furthermore, the gain iscontrolled by auto gain control (AGC) processing, and analog-to-digital(A/D) conversion is performed to output a digital image signal.

The image signal processor 152 performs predetermined signal processingsuch as demosaic processing, white balance adjustment processing, colorcorrection processing, gamma correction processing, Y/C conversionprocessing, auto exposure (AE) processing, and resolution conversionprocessing, on the image signal.

The codec 153 performs, in one example, coding processing for recordingor communication on the image data that is subjected to thepredetermined processing.

The display control unit 154 acquires a focus evaluation value from thephase difference sensor 123, and adjusts and sets a peaking thresholdserving as a threshold for highlighting on the basis of the acquiredfocus evaluation value. The adjustment of the peaking threshold will bedescribed later in detail. In addition, the display control unit 154performs highlighting using the peaking processing on the basis of thepeaking threshold that is set when the energy of the high frequencycomponent for each pixel in an image exceeds the peaking threshold. Thispeaking processing is performed by high frequency component extractionprocessing, comparison processing, and peaking rendering processing. Thehigh frequency component extraction processing extracts a high frequencycomponent by a high-pass filter having a predetermined filter factor,and the comparison processing compares the energy of the extracted highfrequency component with the peaking threshold. The peaking renderingprocessing highlights a pixel at a location corresponding to informationon position of a pixel in which the energy of the high frequencycomponent is determined to be more than the peaking threshold bycomparison. The peaking processing allows a focused subject in an imageto be highlighted. This makes it possible the user to perform focusingeasily by performing focusing so that more peaking markers are rendered.

Moreover, the focus evaluation value may be supplied from the phasedifference sensor 123 to the display control unit 154 via the controlunit 110 or may be directly supplied from the phase difference sensor123 to the display control unit 154.

The display control unit 154 may be configured as a program. Thisprogram may be installed in the image capturing device 100 in advance,or may be delivered using downloading, a storage medium, or the like andinstalled by the user himself. The control unit 110 may execute theprogram so that the control unit 110 functions as the display controlunit 154. Moreover, the display control unit 154 may be implemented notonly by a program but also by a combination of dedicated devices,circuits, or the like with hardware having the above function.

The storage unit 160 is a high capacity storage medium such as a harddisk, Memory Stick (registered trademark of Sony Corporation), and SDmemory card. The image is stored, in one example, in a compressed stateusing the standard such as Joint Photographic Experts Group (JPEG). Inaddition, exchangeable image file format (Exif) data includinginformation on a stored image and additional information such asshooting date and time is also stored in association with the image. Themoving image is saved in a format such as Moving Picture Experts Group2(MPEG2), MPEG 4, or the like.

The display unit 170 is a display device composed of, in one example, aliquid crystal display (LCD), a plasma display panel (PDP), an organicelectro luminescence (EL) panel, or the like. On the display unit 170, auser interface, a menu screen, and a monitoring image being captured ofthe image capturing device 100, a captured image and a captured movingimage recorded in the storage unit 160, or the like are displayed.

The input unit 180 includes, in one example, a power button forswitching power on/off, a release button for instructing the start ofrecording of a captured image, an operating tool for zoom adjustment, atouch screen integrally formed with the display unit 170, or the like.When the input unit 180 receives an input, a control signalcorresponding to the input is generated and is output to the controlunit 110. Then, the control unit 110 performs arithmetic processing andcontrol corresponding to the control signal.

The camera shake sensor 190 detects camera shake while shooting, in oneexample, by an acceleration sensor or an angular velocity sensor in twoaxial directions, and supplies camera shake information to the controlunit 110. The control unit 110 performs camera shake correction controlon the basis of the camera shake information from the camera shakesensor.

The image capturing device 100 equipped with a function as the displaycontrol device is configured as described above.

[1-2. Peaking Processing]

The peaking processing performed in the image capturing device 100 isnow described. An overview of peaking is described with reference toFIG. 2. Peaking is processing of detecting a high frequency component inan image, specifying an in-focus portion of a subject, and highlightingpixels constituting an edge portion (e.g., pattern or contour) of thesubject. The pixels constituting the edge portion are rendered with amarker of a predetermined color, and so the pixels are highlighted byincreasing the line along the edge portion of the subject or bythickening the line along the contour of the subject. In addition, itmay be performed by changing the luminance or color tone of a pixel, orsuperimposing a signal for highlighting on the pixel. The luminance andcolor tone of a pixel other than the pixel to be highlighted may berelatively reduced so that the pixel other than the pixel to behighlighted is displayed by blurring. The display mode is not limited toa particular method as long as the pixel to be highlighted can bedistinguished from other pixels.

FIGS. 2A to 2D illustrate examples of images constituting a monitoringimage to be subjected to the peaking processing. In FIG. 2A, a frame A,which is indicated by a broken line and superimposed on an image, is anarea to be detected by the phase difference sensor 123. FIG. 2Billustrates an example in which the peaking processing is performed bysetting an appropriate peaking threshold for the image. It is possibleto show a flower as a subject, which is in focus, to the user bydepicting a peaking marker on pixels constituting the image and drawinga thick line along the entire contour of the flower.

FIG. 2C illustrates a case where the peaking processing is performed onthe image and the peaking threshold is set to be low unsatisfactorily.In this case, the peaking marker is depicted on pixels constituting theimage, and the thick lines along the contours of objects constitutingthe blurred background as well as the flower as a main subject arerepresented. Thus, it fails to show appropriate focusing to the user.

FIG. 2D illustrates a case where the peaking processing is performed onthe image and the peaking threshold is set to be high unsatisfactorily.In this case, a peaking marker does not appear at the portion wherecontrast is low in the flower serving as the main subject in the image,so it fails to show appropriate focusing to the user. Thus, in thepeaking processing, it is important to set an appropriate peakingthreshold.

Next, the procedure of the peaking processing in the first embodiment isdescribed with reference to the flowchart of FIG. 3. First, in step S11,the display control unit 154 acquires a focus evaluation value in imagesconstituting the monitoring image to be processed from the phasedifference sensor 123. As described above, the focus evaluation value isa value that represents blur in the image, and in one example,represents the radius of the magnitude of the PSF in pixel units. As themagnitude of blur is smaller, the subject is more in focus. Next, instep S12, the display control unit 154 calculates an adjustment factorfrom the focus evaluation value. The adjustment factor is a coefficientused to set the peaking threshold depending on the proximity to a focusbest condition indicated by the focus evaluation value. A method ofcalculating the adjustment factor is described below.

FIG. 4 is a graph illustrating the relationship between the adjustmentfactor used to set the peaking threshold and the focus evaluation value.In the graph, the vertical axis represents the adjustment factor and thehorizontal axis represents the magnitude of blur in pixel units, whichis indicated by the focus evaluation value. In the graph, factorcharacteristics, which are used to determine the adjustment factor fromthe focus evaluation value, is indicated by a line segment.

The factor characteristics are set so that the adjustment factorincreases as the focus evaluation value increases. In addition, in theexample shown in the graph of FIG. 4, the factor characteristics are setso that the adjustment factor is fixed at 0.7 in a range where themagnitude of blur that is indicated by the focus evaluation value is twopixels or less. This is the lower limit of the adjustment factor toreduce noise in the focus evaluation value. The lower the adjustmentfactor, the lower the peaking threshold, and so the peaking marker tendsto increase but tends to depend on noise. Thus, it is intended toprevent the peaking threshold from being less than a predeterminedvalue. In addition, these two pixels correspond to limit valuesdetermined by the detection performance of the magnitude of blur that isobtained from the phase difference sensor. When the focus point at whichthe high frequency component in the image has the maximum energy comesclose to the in-focus state, the S/N of the focus evaluation valuedecreases due to the performance and detection accuracy of the phasedifference sensor 123, which leads to impairment of stability of values.This is the limit of the focus information obtained by the phasedifference sensor 123.

Further, the factor characteristics are set so that the adjustmentfactor increases linearly as the magnitude of blur indicated by thefocus evaluation value increases from two to eight pixels. Then, thefactor characteristics are set so that the adjustment factor is 1.0 whenthe focus evaluation value is eight pixels. These eight pixels indicatethe limit value of the range where the user is allowed to indicate thepeaking marker, and correspond to the predetermined value of the focusevaluation value in the claims. In a case where the magnitude of blur isequal to or more than the limit value of this allowable range, it isdetermined that the main subject in the image is out of focus at all andso the peaking processing is not performed. In a case where themagnitude of blur is equal to or less than the predetermined value, thepeaking processing is performed assuming that the subject is in focus.

The peaking threshold is calculated by multiplying the maximum value ofa high frequency component within an area detected by the phasedifference sensor 123 (hereinafter referred to as a phase differencesensor detection area) in the image by the adjustment factor. Thepeaking processing is performed in the case where the maximum value ofthe high frequency component in the image is equal to or more than thepeaking threshold. In a case where the adjustment factor is a value of 1or more, the peaking threshold is more than the maximum value of thehigh frequency component within the phase difference sensor detectionarea. Then, the pixel detected by the peaking processing disappears andthe peaking threshold does not substantially function, thus it isdesirable to set the factor characteristic so that the adjustment factoris 1 at the allowable limit value of the magnitude of blur indicated bythe focus evaluation value.

Moreover, the specific values of eight pixels or two pixels, which arethe magnitude of blur and the adjustment factor of 0.7 as the focusevaluation value in the graph of FIG. 4, are merely examples, and thevalues are not limited to these values. The value of a pixel that is themagnitude of blur also varies depending on types of a camera, the sizeof an image, or the like.

Referring back to the description of the flowchart of FIG. 3, in stepS13, the display control unit 154 acquires the maximum value of a highfrequency component in the phase difference sensor detection area in theimage. Moreover, the maximum value of the high frequency component isnot necessarily a value within substantially the same range as the phasedifference sensor detection area, and may be a value in a wider rangethan a phase difference sensor detection area including the phasedifference sensor detection area therein. The maximum value of the highfrequency component in the phase difference sensor detection area isobtainable from, in one example, a histogram of the frequency of theimage. In addition, in one example, in autofocus mode, the highfrequency component is obtainable as a numerical value, so theacquisition of the numerical value makes it possible to obtain themaximum value of the high frequency component.

Then, in step S14, the display control unit 154 calculates a peakingthreshold by multiplying the maximum value of the high frequencycomponent by the adjustment factor. This calculated peaking threshold isset as a threshold used for the peaking processing, and highlighting bythe peaking processing is performed in the case where the energy of thehigh frequency component for each pixel in the image exceeds the peakingthreshold.

Next, an operation of the peaking processing in the first embodiment isdescribed with reference to FIGS. 5A to 5C. FIG. 5A illustrates avariation of focus evaluation values with a change in focus positions,and the vertical axis represents the focus evaluation value. FIG. 5Billustrates the high frequency component and the peaking threshold inthe image, and the vertical axis represents the maximum value of thehigh frequency component in the phase difference sensor detection area.In FIG. 5B, the thick line represents the high frequency component inthe image, and the thin line represents the peaking threshold. FIG. 5Cillustrates the number of peaking markers within an in-focus near range,and the vertical axis represents the number of peaking markers in thephase difference sensor detection area. In FIGS. 5A, 5B, and 5C, thehorizontal axis represents the amount of deviation from the best focusposition, and “0” on the horizontal axis represents the best focusposition.

In FIGS. 5A, 5B, and 5C, a range sandwiched between a pair of brokenlines indicates the in-focus near range. In the present technique, thein-focus near range includes a focus position in which high frequencycomponents in the image have the maximum energy, and the in-focus nearrange is a range where the magnitude of blur in the image indicated bythe focus evaluation value is a predetermined value or less. In the casewhere the in-focus state is within the in-focus near range, it can besaid that the fact that the focus is near the in-focus range is knownfrom the focus evaluation value by the phase difference sensor 123. Inaddition, the focus evaluation value indicates the magnitude of blur inthe image, so the decrease in the focus evaluation value indicates anapproach within the in-focus near range, that is, the focus matches thesubject. In the case where the in-focus state is within the in-focusnear range, highlighting is performed using the peaking marker, so it ispossible for the user to easily focus on the position having the highestenergy of the high frequency component in the image.

As illustrated in FIG. 5A, there may be a case where the focus isadjusted, the magnitude of blur in the image that is indicated by thefocus evaluation value decreases, and the focus is close to a focusposition having the highest energy of the high-frequency component inthe image within the in-focus near range. In this case, the focusevaluation value has an error depending on the performance and detectionaccuracy of the phase difference sensor 123, which leads to the decreasein reliability of the value. This refers to a state, in one example,where the focus evaluation value described with reference to FIG. 4reaches two pixels.

In FIG. 5A, the alternate long and short dash lines represent apredetermined value of the focus evaluation value, and the predeterminedvalue indicates the limit value of the allowable range of the magnitudeof blur indicated by the focus evaluation value. This corresponds toeight pixels in FIG. 4. Thus, two points where the line segmentindicating the focus evaluation value intersects the limit value of theallowable range of the focus evaluation value are both ends of thein-focus near range. As described above, the in-focus near range isdetermined using the focus evaluation value from the phase differencesensor 123.

As described above, the peaking threshold is calculated by multiplyingthe maximum value of the high frequency component by the adjustmentfactor. The adjustment factor is a value of 1 or more outside thein-focus near range, so the peaking threshold is more than the maximumvalue of the high frequency component outside the in-focus near range asillustrated in FIG. 5B. As long as the peaking threshold is more thanthe high frequency component, the peaking marker is not displayed andhighlighting is not performed.

On the other hand, when the focus evaluation value is a value equal toor less than the limit value of the allowable range, the adjustmentfactor becomes a value of 1 or less, so the peaking threshold is lessthan the maximum value of the high frequency component within thein-focus near range as illustrated in FIG. 5B. This allows the peakingmarker to be tend to increase in the in-focus near range around thefocus position where the high frequency component in the image has themaximum energy, and as illustrated in FIG. 5C, the peaking markerappears within the in-focus near range. The decrease of the peakingthreshold increases the maximum value of the high frequency componentthat is equal to or more than the peaking threshold, so the number ofappearing peaking markers increases. Then, the number of peaking markersincreases as approaching the maximum value of the high frequencycomponent, but the number of peaking markers decreases as deviating fromthe maximum value of the high frequency component.

In this manner, the in-focus near range is determined using the focusevaluation value from the phase difference sensor 123, and the peakingmarker is displayed only within the in-focus near range. Thus, it ispossible to display typically the peaking marker only within thein-focus near range around the focus position at which thehigh-frequency component in the image has the maximum energy, withoutadjusting settings for peaking processing such as peaking threshold,regardless of subject type or shooting conditions.

The first embodiment allows the peaking threshold to be determinedautomatically so that the peaking marker appears in the in-focus nearrange serving as the range in which the fact that the focusing isachieved to some extent using the focus evaluation value by the phasedifference sensor 123 is detected. This appearance of the peakingthreshold is based on the focus evaluation value, regardless of subjecttype or shooting conditions. In addition, the peaking processing isperformed within the in-focus near range regardless of whether thecontrast of a subject is high or low. Thus, it is possible to implementthe processing of typically displaying the peaking marker within thein-focus near range using simple processing without adjusting settingsfor the peaking processing.

2. Second Embodiment

Next, a second embodiment of the present technology is described.Moreover, the configuration of the image capturing device 100 is similarto that of the first embodiment, so the description thereof is omitted.The second embodiment changes settings of the peaking thresholddepending on whether the in-focus state is within the in-focus nearrange or outside of the in-focus near range, and further maintains thepeaking threshold at a fixed value within the in-focus near range.

[2-1. Peaking Processing]

The procedure of the peaking processing according to the secondembodiment is described with reference to the flowchart of FIG. 6.Moreover, the same procedure step number denotes processing similar tothat in the flowchart of the first embodiment.

First, in step S21, a focus evaluation value in each of a current frameimage (hereinafter referred to as a current frame) constituting amonitoring image to be processed and a frame image immediately precedingtemporally the current frame (hereinafter referred to as a previousframe) is acquired from the phase difference sensor 123. Moreover, thefocus evaluation value acquired in the previous frame may be stored in amemory or the like and then the focus evaluation value in the currentframe may be acquired.

Next, in step S22, the in-focus state is determined from the focusevaluation value. The determination of the in-focus state is performedby determining whether the focus position is within the in-focus nearrange or outside of the in-focus near range. The determination of thein-focus state from the focus evaluation value can be performed bydetermining whether the magnitude of blur indicated by the focusevaluation value is equal to or less than a predetermined value. Thispredetermined value indicates the limit value of the allowable range todisplay the marker indicated by the focus evaluation value. In a casewhere the magnitude of blur is equal to or less than a secondpredetermined value, a main subject is in focus and it is subjected tothe highlighting processing by the peaking marker. In the case where themagnitude of blur is equal to or less than the predetermined value, thein-focus state is within the in-focus near range. However, in a casewhere the magnitude of blur is equal to, or more than, the predeterminedvalue, the in-focus state is outside of the in-focus near range.

Next, in step S23, it is determined whether the in-focus state isoutside of the in-focus near range in the previous frame and thein-focus state is within the in-focus near range in the current frame.In a case where the in-focus state when proceeding from the previousframe to the current frame is changed from outside of the in-focus nearrange to within the in-focus near range, the processing proceeds to stepS12 (Yes in step S23). Moreover, the previous frame does not exist in afirst frame, and so the in-focus state of the previous frame is treatedas undefined. This allows the determination of whether there is a changefrom outside of the in-focus near range to within the focusing nearrange to be Yes.

Next, in step S12, the display control unit 154 calculates theadjustment factor from the focus evaluation value. The adjustment factoris calculated in a similar manner to the first embodiment. Next, in stepS13, the display control unit 154 acquires the maximum value of the highfrequency component in the phase difference sensor detection area in theimage. Then, in step S14, the display control unit 154 calculates thepeaking threshold by multiplying the maximum value of the high frequencycomponent by the adjustment factor, and sets the peaking threshold as athreshold used for the peaking processing. The highlighting by thepeaking processing is performed on the basis of the peaking threshold inthe case where the energy of the high frequency component for each pixelin the image exceeds the peaking threshold.

The description returns to step S23. If it is not determined in step S23that the in-focus state when proceeding from the previous frame to thecurrent frame is changed from outside of the in-focus near range towithin the in-focus near range, the processing proceeds to step S24 (Noin step S23). Next, in step S24, it is determined whether the in-focusstate when proceeding from the previous frame to the current frame ischanged from within the in-focus near range to outside of the in-focusnear range. If it is determined that the in-focus state is changed fromwithin the in-focus near range to outside of the in-focus near range,the processing proceeds to step S25 (Yes in step S24).

Then, in step S25, processing for returning the peaking threshold to theinitial value is performed. This initial value is a value more than themaximum value of the high frequency component in the phase differencesensor detection area. By setting the peaking threshold to be a valuemore than the maximum value of the high frequency component, peaking isnot performed outside the in-focus near range, so the peaking markerdoes not appear. This allows the peaking marker to appear only withinthe in-focus near range, and it is easier for the user to focus usingpeaking.

On the other hand, if it is not determined in step S24 that the in-focusstate is changed from within the in-focus near range to outside of thein-focus near range, the processing is terminated without performing newprocessing (No in step S24). If shooting continues, the processing isrepeated again from step S21. The case where it is not determined instep S24 that the in-focus state is changed from within the in-focusnear range to outside of the in-focus near range refers to one of a casewhere the in-focus state remains within the in-focus near range or acase where the in-focus state remains outside the in-focus near range.In other words, the peaking threshold remains without setting orinitializing the peaking threshold.

Next, an operation of the peaking processing in the second embodiment isdescribed with reference to FIGS. 7A to 7C. FIG. 7A is a diagram similarto FIG. 5A in the first embodiment, and illustrates a variation of thefocus evaluation value with the change in focus position. FIG. 7B is adiagram similar to FIG. 5B in the first embodiment, and illustrates thehigh frequency component and the peaking threshold in the image. In FIG.7B, the thick line represents the high frequency component and the thinline represents the peaking threshold. FIG. 7C is a diagram similar toFIG. 5C in the first embodiment, and illustrate the number of peakingmarkers within the in-focus near range. In FIGS. 7A, 7B, and 7C, a rangesandwiched between a pair of broken lines indicates the in-focus nearrange.

As described above in the description of the flowchart, in the secondembodiment, a new peaking threshold is set in the case where thein-focus state is changed from outside of the in-focus near range towithin the in-focus near range. Then, the peaking threshold remains aslong as the in-focus state remains within the in-focus near range(continuing to exist within the in-focus near range), and thehighlighting by the peaking processing is performed in the case wherethe energy of the high frequency component for each pixel in the imageexceeds the peaking threshold.

Thus, as illustrated in FIG. 7B, even if the value of the high frequencycomponent varies within the in-focus near range, there is no change inthe peaking threshold. Thus, as illustrated in FIG. 5B of the firstembodiment, the maximum value of the high frequency component is higharound the focus position where the high frequency component in theimage has the maximum energy, so there is no possibility that thepeaking threshold is raised.

Thus, the number of peaking markers increases without decreasing theincrease pace of the number of peaking markers near the focus positionwhere the high frequency component in the image has the maximum energyas illustrated in FIG. 7C, as compared with the graph of FIG. 5C of thefirst embodiment. This makes it easy to recognize visually a change inthe quantity of the peaking markers near the focus position where thehigh frequency component in the image has the maximum energy, so theuser can more easily perform focusing.

In the case where the in-focus state is changed from within the in-focusnear range to outside of the in-focus near range, the peaking thresholdis initialized in step S25. Then, in the case where the in-focus stateremains outside the in-focus near range, the processing from step S12 tostep S14 is not performed, so the peaking threshold remains at theinitial value regardless of the high frequency component. This preventsunnecessary peaking marker from being displayed in the case of deviatingfrom the in-focus near range, so the user is more likely to concentrateon the focusing operation.

In the first embodiment, as illustrated in FIGS. 4 and 5A, when thefocus evaluation value is fixed by clipping the focus evaluation valuedepending on the detection accuracy of the phase difference sensor 123,the adjustment factor also is fixed. Then, even when the adjustmentfactor is fixed, the maximum value of the high frequency componentincreases, so the peaking threshold is slightly larger in the clippedrange. As the peaking threshold increases, the frequency of appearanceof peaking markers decreases.

However, according to the second embodiment, it is possible to maintaintypically the peaking threshold at a fixed value within the in-focusnear range. Thus, the frequency of appearance of peaking markers doesnot decrease by increasing the peaking threshold, which has decreasedwithin the in-focus near range, near the focus position where the highfrequency component in the image has the maximum energy. This makes iteasier to focus by using peaking.

3. Third Embodiment

Next, a third embodiment of the present technology is described.Moreover, the configuration of the image capturing device 100 is similarto that of the first embodiment, and so the description thereof isomitted. As illustrated by a plurality of frames surrounded by brokenlines, which are superimposed on the image in FIG. 8A, the thirdembodiment differs from the first and second embodiment in that thephase difference sensor 123 has a detection area for the focusevaluation value by the plurality of phase difference sensors 123 in theimage. In the example of FIG. 8A, there are 20 phase difference sensordetection areas in total of four vertical columns and five horizontalcolumns.

Furthermore, in the third embodiment, as illustrated by regionssegmented by broken lines superimposed on the image in FIG. 8B, it issegmented into a plurality of regions so that each region corresponds toeach of the plurality of phase difference sensor detection areas. Eachof the plurality of regions has the peaking threshold, and the peakingprocessing is performed by setting the peaking threshold for each of theregions. Moreover, the number of phase difference sensor detection areasis merely an example, and it may be more or less than that number.

[3-1. Peaking Processing]

The procedure of the peaking processing according to the thirdembodiment is described with reference to the flowchart of FIG. 9.Moreover, the same procedure step number denotes processing similar tothat in the flowchart of the first and second embodiments. In theprocessing of the flowchart illustrated in FIG. 9, steps S31 and S32 areperformed for all the plurality of phase difference sensor detectionareas existing in the image, and each step other than steps S31 and S32is performed for one of the plurality of phase difference sensordetection areas.

First, in step S31, the focus evaluation value in each of the currentframe constituting the monitoring image to be processed and in theprevious frame, which is the frame immediately preceding the currentframe, is acquired from the phase difference sensor 123. Next, in stepS32, the in-focus state is determined from the focus evaluation value.

Next, in step S33, subjects in the phase difference sensor detectionareas in the current frame constituting the monitoring image and theprevious frame that is the temporally previous frame of the currentframe are compared, and areas where the same subject is detected areassociated with each other. Thus, the phase difference sensor detectionarea in the previous frame corresponding to one phase difference sensordetection area in the current frame is specified, and the focusevaluation values in one phase difference sensor detection area in thecurrent frame and in the corresponding phase difference sensor detectionarea in the previous frame are associated with each other. This enablesstable operation even when the position of the subject varies in thescreen.

The comparison between the subjects in the phase difference sensordetection areas of the current frame and the previous frame can beperformed using known subject recognition processing, matchingprocessing, or the like. An example of the recognition method mayinclude an object recognition technique based on template matching, amatching method based on luminance distribution information of asubject, and a method based on a skin color portion included in animage, a feature amount of a human face, or the like. In addition, thesetechniques may be combined to increase the accuracy of recognition.

Next, in step S34, it is determined whether the in-focus state whenproceeding from the previous frame to the current frame is changed fromoutside of the in-focus near range to within the in-focus near range. Inthe case where it is changed from outside of the in-focus near range towithin the in-focus near range, the processing proceeds to step S12 (Yesin step S34). Moreover, the previous frame does not exist in the firstframe, so the in-focus state of the previous frame is treated asundefined. In addition, the in-focus state of the area that failed toassociate is also treated as undefined. Thus, the determination ofwhether the in-focus state is changed from outside of the in-focus nearrange to within the in-focus near range is set to Yes.

Next, in step S12, the display control unit 154 calculates theadjustment factor from the focus evaluation value. Next, in step S13,the display control unit 154 acquires the maximum value of the highfrequency component in the phase difference sensor detection area in theimage. Then, in step S14, the display control unit 154 calculates apeaking threshold by multiplying the maximum value of the high frequencycomponent by the adjustment factor, and sets the peaking threshold asthe threshold used for the peaking processing. Then, the highlighting bythe peaking processing is performed in the case where the energy of thehigh frequency component for each pixel in the image exceeds the peakingthreshold.

The description returns to step S34. If it is not determined in step S34that the in-focus state in the current frame is changed from outside ofthe in-focus near range to within the in-focus near range, theprocessing proceeds to step S35 (No in step S34). Next, in step S35, itis determined whether the in-focus state when proceeding from theprevious frame to the current frame is changed from within the in-focusnear range to outside of the in-focus near range. If it is determinedthat the in-focus state is changed from within the in-focus near rangeto outside of the in-focus near range, the processing proceeds to stepS25 (Yes in step S35).

Then, in step S25, the processing for returning the peaking threshold tothe initial value is performed. This processing is similar to that ofstep S25 in the second embodiment. This initial value is larger than themaximum value of the high frequency component in the phase differencesensor detection area. The peaking threshold, which is set to a valuelarger than the maximum value of the high frequency component, preventsthe peaking processing being performed outside the in-focus near rangeand prevents the peaking marker from appearing. This allows the peakingmarker to appear only in the case the in-focus state is within thein-focus near range, so it is easier for the user to focus usingpeaking.

On the other hand, if it is not determined in step S35 that the in-focusstate is changed from within the in-focus near range to outside of thein-focus near range, the processing proceeds to step S36 (No in stepS35). In the case where it is not determined in step S35 that thein-focus state is changed from within the in-focus near range to outsideof the in-focus near range, the in-focus state when proceeding from theprevious frame to the current frame remains within the in-focus nearrange or remains outside the in-focus near range.

In step S36, the peaking threshold used in one phase difference sensordetection area of the previous frame is set as the peaking threshold ofthe phase difference sensor detection area in the current framecorresponding thereto. This allows the peaking threshold to remain inboth of one phase difference sensor detection area in the previous frameand a phase difference sensor detection area in the current framecorresponding thereto. When the energy of the high frequency componentfor each pixel in the image exceeds the peaking threshold, thehighlighting by the peaking processing is performed.

If shooting continues, the processing is repeated again from step S33.The processing described above is processing for one of the plurality ofphase difference sensor detection areas in the image. Then, thisprocessing is performed for all phase difference sensor detection areas.

In one example, there may be a case where a subject moves or the camerashakes during focusing. In this case, the position of the subject mayvary between the previous frame and the current frame. In FIG. 10, FIG.10A illustrates an example of a previous frame, and FIG. 10B illustratesan example of a current frame. In the example of FIG. 10, due to theshaking of the camera, stamens of the flower are located in the phasedifference sensor detection area A of the previous frame in FIG. 10A,but in the current frame shown in FIG. 10B, the position of the stamensof the flower are shifted and the stamens of the flower are located inthe phase difference sensor detection area B. In this case, the subjectin the phase difference sensor detection area A is changed between theprevious frame and the current frame, and so when the phase differencesensor detection area A is compared between the previous frame and thecurrent frame, the focus evaluation value does not correspond. If thefocus evaluation value does not correspond, the peaking processing failsto be performed and setting of the appropriate peaking threshold isfailed.

However, according to the third embodiment, the subjects in the phasedifference sensor detection areas between the previous frame and thecurrent frame are compared and the areas are associated with each other.Thus, in the example of FIG. 10, the focus evaluation values of thephase difference sensor detection area A and the phase difference sensordetection area B are associated with each other. This allows a series ofprocessing to be performed in the phase difference sensor detection areaA of the previous frame in FIG. 10A and the phase difference sensordetection area B of the current frame in FIG. 10B. Thus, even when thephase difference sensor 123 has a plurality of detection areas, thepeaking threshold is appropriately set in the current frame for each ofthe plurality of areas, and the peaking processing is performed for eacharea.

4. Modified Example

Although the embodiments of the present technology are described indetail above, the present technology is not limited to theabove-described embodiments, and various modifications based on thetechnical idea of the present technology are possible.

The characteristics for determining the adjustment factor may beadjusted or may be selected depending on the user's preference or thelike. In one example, in a case where the necessary accuracy of focusingdiffers depending on the use of the image or the like, it is desirableto change the adjustment characteristics accordingly.

In one example, in the 4K resolution video system (hereinafter referredto as 4K) and the video graphics array (VGA), 4K has higher resolutionand 4K is typically necessary to have focusing with higher accuracy thanVGA. In this case, in 4K where high-accuracy focusing is necessary, thelimit value of the allowable range of the magnitude of blur is less thanthe VGA, so the allowable range is narrow. Thus, the factorcharacteristics of 4K illustrated in FIG. 11A has a sharper inclinationangle than the factor characteristics of VGA illustrated in FIG. 11B.

Furthermore, in the case of displaying an image with a high resolutionin a large mode or printing it in a large mode, it is typicallynecessary that the camera is focused with high accuracy, the limit valueof the allowable range of the magnitude of blur is small, and so theallowable range is narrow. Thus, the inclination angle of the factorcharacteristics becomes steep.

Further, a user with high shooting skills generally has high sensitivityto blurring due to out-of-focus and pursues more precise focusing. Thus,there is a tendency to prefer characteristics that the peaking responsehardly occurs (characteristics having a narrow display allowable rangeof the peaking marker). In a case where such a user uses it, the limitvalue of the allowable range of the magnitude of blur indicated by thefocus evaluation value is set to a small value, and the inclinationangle of the characteristics for obtaining the adjustment factor is setto be large. If the limit value of the allowable range of the magnitudeof blur is set to be a small value and the angle of the inclination ofthe characteristics is set to be large, the range of the in-focus nearrange is narrow and the peaking marker hardly appears.

Further, it is difficult to focus the moving image shooting as comparedwith the still image shooting, so it is considered that the peakingresponse is less likely to occur. In such a case, if the limit value ofthe allowable range of the phase difference sensor 123 is set to a smallvalue and the inclination angle of the characteristics for obtaining theadjustment factor is made large as described above, the peaking markerhardly appears, and it can make the peaking response difficult.

Furthermore, under shooting conditions with a small exposure amount,noise easily occurs on the magnitude of blur indicated by the focusevaluation value. Thus, depending on the amount of noise, by increasingthe lower limit of the adjustment factor as the noise increases, it ispossible to make peaking response less likely occur to reduce thefluctuation of behavior.

The factor characteristics for determining the adjustment factor are notnecessarily linear as illustrated in FIGS. 4 and 11, and the factorcharacteristics may be curve-like characteristics.

Further, the peaking threshold is calculated by multiplying the factorcharacteristics by the maximum value of the high frequency component inthe embodiment, but other values of the high frequency component may beused. An example of other values includes an average value, an uppervalue (e.g., top 10%, etc.), or the like of high frequency components.The use of such a value allows the stability of the peaking processingto be increased.

Further, the peaking processing is performed by performing highlightingdepending on the high frequency component in the embodiment.Alternatively, the peaking processing may be performed depending on theexposure. In this case, in one example, the peaking threshold may be setfrom the exposure value (EV) value and the adjustment factor, or theF-number of the lens 121, the shutter speed, or the like may be used.

Further, the peaking processing may be performed by performinghighlighting depending on the specific state of a subject. The specificstate of the subject is, in one example, a value indicating thebrightness and roughness of the subject when the subject is a person. Inthis case, the value is compared with a predetermined value to determinethe skin condition, and so the highlighting depending on the skincondition may be performed.

Techniques for focusing an image after shooting rather than duringshooting are becoming widespread. In one example, it is possible toperform it by acquiring a series of image groups composed of a pluralityof images with different focuses at the time of shooting and byselecting an image with a desired in-focus state from among the imagegroups using a personal computer or the like after shooting. Inaddition, there is also a technique for associating depth informationcalled depth map with images and for using the depth map. Furthermore,there is a technique, which is called light field, for obtaining variousfocused images without changing the focus of the main lens by arranginga plurality of micro-lenses having different focuses on the frontsurface of the image sensor of the image capturing device and byreconstructing the focused image after shooting.

The present technology is applicable to techniques for focusing an imageafter shooting as described above. In one example, when acquiring aseries of image groups composed of a plurality of images with differentfocuses, by further associating a focus evaluation value withinformation on a high frequency component for each image, it is possibleto display the peaking marker by calculating the adjustment factor andthe peaking threshold after shooting.

A technique, which is called light detection and ranging or laserimaging detection and ranging (LIDAR), for detecting scattered light byirradiating a laser emitting pulsed light and for calculating thedistance from the reflection time (the time from emission to detectionof reflected light) may be applicable to the present technology.

Further, a technique, which is called time of flight (ToF), forobtaining the traveling time (delay time) of the light until the lightemitted from a light source is reflected by an object and reaches thesensor and for obtaining a distance to the subject from the speed of thelight may be used.

In highlighting using the peaking marker, the color of the peakingmarker may vary for each frame. In one example, serial numbers areallocated to all frames of a series of image groups constituting amonitoring image, and in one example, the peaking marker is set to begreen for odd frames and to be red for even frames. The change in thecolor of the peaking marker for each frame makes it possible to preventoccurrence of a situation in which the peaking marker becomes similarcolor to the subject and the peaking is hard to distinguish.

Further, the user may optionally select the color of the peaking marker.

Furthermore, it is also possible to highlight the focused subject bytechniques for changing or superimposing the luminance of the focusedsubject, for changing or superimposing the color of the subject, forreducing relatively the color tone of a subject other than the focusedsubject, or the like.

The easiness or difficulty of increasing the number of peaking markerscan be made by adjusting the center frequency and/or the peak width ofthe peak of the high-pass filter for the high frequency component in theimage on the basis of the focus evaluation value. In the case of asubject having a low high frequency component, peaking markers hardlyappear in some cases even if the maximum value of the high frequencycomponent is compared with the peaking threshold. In that case, thepeaking marker may be easily increased by lowering the center frequency.

Further, in a case where the maximum value of the high frequencycomponent falls below the predetermined value, the processing may beperformed after increasing the maximum value of the high frequencycomponent to a specific value.

Further, if the peaking threshold is low unsatisfactorily (peakingmarker is likely to increase), the peaking threshold tends to respond tonoise, so the peaking threshold may be restricted so that the peakingthreshold does not fall below the preset value.

Further, in order to be able to support even a smooth subject with asmall high frequency component, a plurality of kinds of high-passfilters based on the peaking threshold are prepared, and a filter to beused among them may be determined when determining the adjustmentfactor.

There may be a case where a high frequency component does not appeardepending on the type of the lens even if the image is in focus due toaberration around the image. Thus, the characteristics of the high-passfilter based on the peaking threshold may be changed depending on thetype of the lens attached to the image capturing device.

Further, as shown in the image capturing device 200 according to themodified example illustrated in FIG. 12, the present technology is alsoapplicable to an image capturing device, which is equipped with aso-called image plane phase difference AF function and which includesthe phase difference sensor 141 provided on the image sensor 140.Moreover, in the image capturing device 200 equipped with the imageplane phase difference AF function, it is unnecessary to guide incidentlight to the phase difference sensor in a direction different from thatof the image sensor 140, so a half mirror is unnecessary.

Additionally, the present technology may also be configured as below.

(1)

A display control device including:

a display control unit configured to control display, on a display unit,of highlighting corresponding to a state of an image depending on anin-focus state of a subject in the image being within a predeterminedin-focus near range.

(2)

The display control device according to (1),

in which the in-focus state is based on a magnitude of blur indicated bya focus evaluation value in the image.

(3)

The display control device according to (2),

in which a highlighting threshold is set on the basis of the focusevaluation value and a predetermined value of a high frequency componentin the image, and the highlighting is performed in a case where thepredetermined value of the high frequency component is equal to or morethan the highlighting threshold.

(4)

The display control device according to (2) or (3),

in which the highlighting threshold is set to a value equal to or lessthan the predetermined value of the high frequency component in a casewhere the focus evaluation value is equal to or less than apredetermined value.

(5)

The display control device according to (3) or (4),

in which the highlighting threshold is set to a value equal to or morethan the predetermined value of the high frequency component in a casewhere the focus evaluation value is equal to or more than apredetermined value.

(6)

The display control device according to any of (3) to (5),

in which the predetermined value of the high frequency component is amaximum value of the high frequency component.

(7)

The display control device according to any of (3) to (5),

in which the predetermined value of the high frequency component is anaverage value of the high frequency component.

(8)

The display control device according to any of (3) to (5),

in which the predetermined value of the high frequency component is anupper value of the high frequency component.

(9)

The display control device according to any of (3) to (8),

in which the highlighting threshold is set from the focus evaluationvalue and the predetermined value of the high frequency component in theimage in a case where the in-focus state is changed from outside of anin-focus near range to within the in-focus near range in a current frameand a previous frame, the current frame being a present frame, theprevious frame being a temporally previous frame of the current frame.

(10)

The display control device according to any of (3) to (8),

in which the highlighting threshold is set to a value equal to or morethan the predetermined value of the high frequency component in a casewhere the in-focus state is changed from within the in-focus near rangeto outside of the in-focus near range between the previous frame and thecurrent frame.

(11)

The display control device according to any of (3) to (8),

in which the highlighting threshold used in the previous frame is set asthe highlighting threshold of the current frame in a case where thein-focus state is not changed between the previous frame and the currentframe.

(12)

The display control device according to any of (3) to (8),

in which the focus evaluation value is acquired in a plurality of areasin the image, and

the focus evaluation value is associated for each of the areas betweenframes by comparing the plurality of areas in a current frame that is apresent frame and a previous frame that is a temporally previous frameof the current frame.

(13)

The display control device according to any of (3) to (8),

in which the highlighting threshold is set from the focus evaluationvalue and the predetermined value of the high frequency component in theimage in a case where the in-focus state is changed from outside of anin-focus near range to within the in-focus near range in the previousframe and the current frame.

(14)

The display control device according to any of (3) to (8),

in which the highlighting threshold is set to a value equal to or morethan the predetermined value of the high frequency component in a casewhere the in-focus state is changed from within an in-focus near rangeto outside of the in-focus near range in the previous frame and thecurrent frame.

(15)

The display control device according to any of (3) to (8),

in which the highlighting threshold used in the previous frame is set asthe highlighting threshold of the current frame in a case where thein-focus state is not changed between the previous frame and the currentframe.

(16)

The display control device according to any of (1) to (15),

in which the in-focus near range is a range in which a magnitude of blurindicated by the focus evaluation value is equal to or less than apredetermined value.

(17)

The display control device according to (16),

in which determination of whether the in-focus state is within thein-focus near range or outside the in-focus near range is determined onthe basis of the magnitude of blur indicated by the focus evaluationvalue in the image.

(18)

The display control device according to any of (1) to (17),

in which the highlighting is performed by performing processing ofdepicting a marker on a pixel constituting the image to increase a linealong a contour of the subject and/or to thicken the line along thecontour of the subject.

(19)

A display control method including:

controlling display, on a display unit, of highlighting corresponding toa state of an image depending on an in-focus state of a subject in theimage being within a predetermined in-focus near range.

(20)

A display control program causing a computer to execute a displaycontrol method including

controlling display, on a display unit, of highlighting corresponding toa state of an image depending on an in-focus state of a subject in theimage being within a predetermined in-focus near range.

REFERENCE SIGNS LIST

-   100 image capturing device-   154 display control unit

1. A display control device comprising: a display control unitconfigured to control display, on a display unit, of highlightingcorresponding to a state of an image depending on an in-focus state of asubject in the image being within a predetermined in-focus near range.2. The display control device according to claim 1, wherein the in-focusstate is based on a magnitude of blur indicated by a focus evaluationvalue in the image.
 3. The display control device according to claim 2,wherein a highlighting threshold is set on the basis of the focusevaluation value and a predetermined value of a high frequency componentin the image, and the highlighting is performed in a case where thepredetermined value of the high frequency component is equal to or morethan the highlighting threshold.
 4. The display control device accordingto claim 3, wherein the highlighting threshold is set to a value equalto or less than the predetermined value of the high frequency componentin a case where the focus evaluation value is equal to or less than apredetermined value.
 5. The display control device according to claim 3,wherein the highlighting threshold is set to a value equal to or morethan the predetermined value of the high frequency component in a casewhere the focus evaluation value is equal to or more than apredetermined value.
 6. The display control device according to claim 3,wherein the predetermined value of the high frequency component is amaximum value of the high frequency component.
 7. The display controldevice according to claim 3, wherein the predetermined value of the highfrequency component is an average value of the high frequency component.8. The display control device according to claim 3, wherein thepredetermined value of the high frequency component is an upper value ofthe high frequency component.
 9. The display control device according toclaim 3, wherein the highlighting threshold is set from the focusevaluation value and the predetermined value of the high frequencycomponent in the image in a case where the in-focus state is changedfrom outside of an in-focus near range to within the in-focus near rangein a current frame and a previous frame, the current frame being apresent frame, the previous frame being a temporally previous frame ofthe current frame.
 10. The display control device according to claim 9,wherein the highlighting threshold is set to a value equal to or morethan the predetermined value of the high frequency component in a casewhere the in-focus state is changed from within the in-focus near rangeto outside of the in-focus near range between the previous frame and thecurrent frame.
 11. The display control device according to claim 9,wherein the highlighting threshold used in the previous frame is set asthe highlighting threshold of the current frame in a case where thein-focus state is not changed between the previous frame and the currentframe.
 12. The display control device according to claim 3, wherein thefocus evaluation value is acquired in a plurality of areas in the image,and the focus evaluation value is associated for each of the areasbetween frames by comparing the plurality of areas in a current framethat is a present frame and a previous frame that is a temporallyprevious frame of the current frame.
 13. The display control deviceaccording to claim 12, wherein the highlighting threshold is set fromthe focus evaluation value and the predetermined value of the highfrequency component in the image in a case where the in-focus state ischanged from outside of an in-focus near range to within the in-focusnear range in the previous frame and the current frame.
 14. The displaycontrol device according to claim 12, wherein the highlighting thresholdis set to a value equal to or more than the predetermined value of thehigh frequency component in a case where the in-focus state is changedfrom within an in-focus near range to outside of the in-focus near rangein the previous frame and the current frame.
 15. The display controldevice according to claim 12, wherein the highlighting threshold used inthe previous frame is set as the highlighting threshold of the currentframe in a case where the in-focus state is not changed between theprevious frame and the current frame.
 16. The display control deviceaccording to claim 1, wherein the in-focus near range is a range inwhich a magnitude of blur indicated by the focus evaluation value isequal to or less than a predetermined value.
 17. The display controldevice according to claim 16, wherein determination of whether thein-focus state is within the in-focus near range or outside the in-focusnear range is determined on the basis of the magnitude of blur indicatedby the focus evaluation value in the image.
 18. The display controldevice according to claim 1, wherein the highlighting is performed byperforming processing of depicting a marker on a pixel constituting theimage to increase a line along a contour of the subject and/or tothicken the line along the contour of the subject.
 19. A display controlmethod comprising: controlling display, on a display unit, ofhighlighting corresponding to a state of an image depending on anin-focus state of a subject in the image being within a predeterminedin-focus near range.
 20. A display control program causing a computer toexecute a display control method including controlling display, on adisplay unit, of highlighting corresponding to a state of an imagedepending on an in-focus state of a subject in the image being within apredetermined in-focus near range.